When application developers introduce changes into applications (e.g., source code changes associated with applications), such changes often have unexpected consequences. For example, application changes may cause errors or reductions in application performance. Testing applications after changes have been made may be useful for determining whether the changes resolve previously detected errors (i.e., “bugs”), whether the changes have introduced new errors, whether changes cause reductions in various performance metrics associated with the applications, and so forth. Such testing is commonly referred to as “regression testing.”
Unfortunately, various factors such as complexity inherent in modern mobile applications, variability in mobile computing devices, and instability of network resources may introduce unacceptable levels of noise and wide inconsistencies in regression test results for mobile applications, reducing the usefulness of such regression test results in identifying and resolving errors and reductions in performance metrics.